Repository navigation
test: update six tests made stale by the Stage 3 refactor and measured unit costs; ignore slower_repeat on #1385 - #1389
Merged
Conversation
zzylol
force-pushed
the
fix/stale-tests-after-measured-costs
branch
from
November 17, 2026 17:28
f460a20 to
4443c15
Compare
…asured costs made stale - rootless request overhead: check its charge (2,000/s for `1` every 1 ms at 2 ms per request); the latency and capacity half is ignored on #1316 (latency (L) and budgets are outside the selection MILP). - stddev_over_time exact: planned from raw rows as a per-series StdDev aggregate, not refused. - materialized views vs stored_outputs: fixture DAGs predate the current IR wire format and carry no view_coverage; test and fixture deleted. - nested exact sum: with raw data kept, query-time plans are 25x cheaper than a shared chain; the chain reuse is checked with no raw data. - quantile of rate: the KLL shape is inspected with summaries' CPU free. - identical sum(rate): sharing holds; the sum is an exact aggregate, so one shared state, not two.
…er_time unsupported instead of expecting no plan
…he per-grid execution contexts bug
zzylol
force-pushed
the
fix/stale-tests-after-measured-costs
branch
from
November 18, 2026 14:47
4443c15 to
83ac4b6
Compare
…xclusion, not any selection error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
promql_quantile_of_rate_binds_kll_over_rate_accumulatorlooks for a KLL in the plan Stage 3 selects forquantile(0.99, rate(http_requests_total[5m])), but on 1,000 series the exact sort is cheaper (0.0060/s) and is selected.slower_repeat_reuses_faster_sumis ignored on the planner bug Stage 3 does not share a result between queries on nested evaluation grids: the same exact sum every 60 s and every 120 s runs twice #1385.asap-integration-testsgoes from 7 failures to none (one more ignored, Stage 3 does not share a result between queries on nested evaluation grids: the same exact sum every 60 s and every 120 s runs twice #1385), andasap-planner's tests go from 1 failure to none. With test(devtools): planner_fuzz under the measured unit costs: a KLL rollup case; filter-on-group-key cases ignored on #891 #1416 (the threeplanner_fuzzfailures), the whole workspace suite passes.Review guide
main(1abbeaba2, rebased without conflicts). Not stacked.crates/integration-tests/tests/joint_search_equivalence.rs(two tests rewritten, one split out and ignored), thenplan_sharing.rs(nested_exact_sum_reuses_newest_tumbling_windows).materialized_views.rsand the helper incrates/integration-tests/src/lib.rs.Behavior: before → after
One justification per test. Every test failed at
9ad2c5422(#1236),6b4847c9a,14308fd2c(#1355),4cb4110c1(#1363) andca4be93f0(main), with the same assertion.joint_search_equivalence::rootless_request_overhead_respects_capacity_and_latency1every 1 ms at 2 ms per request to be refused formax_latency_ms: 1and for a 1-core cluster. Stage 3 has no latency rows and supports cost model 1 only. Both were decided: latency (L) and resource budgets are out of scope for the selection MILP (#1204 item 5, 2026-11-06), and cost models 2 and 3 returnUnsupportedCostModel. #1316 tracks bounding latency. Today the capacity half passes only because cost model 3 is refused, and the latency half returns a plan.rootless_request_overhead_is_charged(new, passes) checks that the selected cost is exactly the request overhead, 2,000/s for 1,000 requests/s x 2,000,000 operations, with no operator node; the old test is#[ignore]d on #1316 and now asserts a refusal that is notUnsupportedCostModel.joint_search_equivalence::unsupported_exact_operation_never_returns_an_unvalidated_planstddev_over_time(m[5m])(exact) to find no plan. Exact plans of any operator are planned from raw rows; the selected plan isScan → TimeRange(5m) → Aggregate { PerEntity, [StdDev] }, which validates, is exact, and the executor lowers (AggIntent::StdDevincrates/executor/src/physical_planner/mod.rs). Executor support does not restrict planner design.exact_operation_without_a_summary_plans_from_raw_rows: asserts a valid exact plan with aStdDevaggregate.materialized_views::materialized_views_equal_the_stored_outputs_main_listedstored_outputs_before_view_coverage.json) holds DAGs in the IR wire format from before #1226:SummaryAgghasfamily: {"Sketch": [{algorithm, params}, "PerSubpopulationInstance"]}, which no longer deserializes (invalid type: sequence, expected string or map). Since #1236 a view's coverage is the node'sview_coverage, which the planner sets and the fixture lacks, so converting the DAGs cannot make it pass either. It compared the derived views with thestored_outputsfield that was deleted when views were derived from the DAG. Live plans' views are checked bycrates/planner/tests/runtime_install.rs,sql_table_scans.rs,columnar_batches.rsandcrates/planner/src/installable/tests.rs.plan_sharing::nested_exact_sum_reuses_newest_tumbling_windowssum(sum_over_time(m_gauge[1m]))added to the 2m sum (both every 60 s, 10,000 series) to cost at most 0.1% more, by reading the 2m chain's newest window. With raw data kept, both are answered at query time: 0.0857/s together, 0.0590/s for the 2m sum alone. The plan that shares one ingestion-time chain costs 2.159/s under the measured unit costs (#1355, #1363), 25 times more, so no cheaper plan exists and the 45% is each query's own scan window and sum.raw_data_retained: false, the plan has oneSummaryTumblingWindowChainand both roots read it.promql_to_post_asap::promql_quantile_of_rate_binds_kll_over_rate_accumulatorlogical_candidatesreturns the plan Stage 3 selects, not every candidate. Under the default unit costs, the selected plan for this query on 1,000 series is the exactAggregate { Quantile }over the finalized Rate (0.0060/s), so there is no KLL to inspect. Other shape tests already cost summaries' CPU free for this reason (summary_cpu_free, #1355).candidates_usinguses the newlogical_candidates_with_summary_cpu_free(accuracy still enforced). The selected plan is the pinned shape:SummaryEstimate ← SummaryAgg(Kll) ← FinalizeExactAccumulator ← SummaryAgg(Rate).asap-plannersummary_sharing::identical_ungrouped_queries_share_their_producerssum(rate(x[5m]))twice ("rate, then sum"). The plan has one: the per-series Rate accumulator, with the sum over the finalized rates as a plain exactAggregate. Whether the sum is a state or an aggregate is a cost choice between two exact candidates (the same shape the quantile test above shows). The sharing it tests holds: both queries read the same states.left: 1, right: 2) → asserts the shared states and that the plan deploys exactly one query's states.Interfaces
Test-only.
crates/integration-tests/src/lib.rspub mod planning { + /// [`logical_candidates`] with summaries' CPU costed free + /// ([`summary_cpu_free`]) and accuracy enforced: the selected plan reads a + /// summary candidate wherever Stage 1 offers one that meets `accuracy`. + /// For tests of a summary plan's shape. + pub fn logical_candidates_with_summary_cpu_free( + root: Rc<OperatorNode>, + accuracy: AccuracyTarget, + ) -> Vec<Rc<OperatorNode>>; }Usage:
promql_quantile_of_rate_binds_kll_over_rate_accumulatorlowersquantile(0.99, rate(http_requests_total[5m]))atmax_error(0.01), plans it over 1,000 series at 1,000 rows/s with this helper, and gets the KLL plan above (cost 0.00113/s with summaries' CPU free), whose nodes the test then checks one by one.Algorithm changes
None.
Decisions and open questions
plan_sharing::slower_repeat_reuses_faster_sumis ignored on Stage 3 does not share a result between queries on nested evaluation grids: the same exact sum every 60 s and every 120 s runs twice #1385 (a planner bug: execution contexts are one per evaluation grid, so the 120 s copy cannot read the 60 s copy's result at the instants both evaluate; a witness plan costs 0.0590/s against the selected 0.0885/s = 0.0590 + 0.0590 / 2), asexact_coarse_sum_reuses_fine_sumis on Stage 3 keeps a MatAtQuery build whose NotMaterialized sibling avoids a second scan (#923 workload) #1142. Its doc comment derives both costs.Verification
cargo nextest run --workspace --no-fail-fastplanner_fuzztests #1416 handles); with #1416 on top: 1889 passed, 0 failed, 15 skipped. Doctests and clippy (changed crates,-D warnings) clean.cargo test -p asap-planner --test planner -- summary_sharingcargo fmt --all -- --checkNew test:
rootless_request_overhead_is_chargedchecks that a query with no operator node still pays its request overhead.Follow-ups
slower_repeat_reuses_faster_sum).rootless_request_overhead_respects_capacity_and_latency).